Spoken Language Tutorial Dialogue

An existing natural language speech dialogue system will be integrated with an existing mathematics tutor to provide adaptive instruction for grade school children. The resulting dialogue tutor will use general linguistic knowledge, including a fairly complex model of English and taskoriented dialogues, to speak to the student when he or she has trouble solving a problem. The tutor will initiate an interaction to diagnose missing knowledge and provide a customized explanation, select appropriate hints for a particular diagnosed problem based on student gender, cognitive development and student history, and provide a summary to ensure that students have the correct information. This work explores issues such as representation, lexical meaning, dealing with ambiguity, making use of information about context, mixed-initiative planning, and development of a formal model of plans based on explicit objects. Goals of the Research Spoken language interfaces are an essential part of future tutoring systems. We will build a conversationally proficient teaching assistant and demonstrate spoken language systems, providing a research platform for issues in natural language understanding, mixed-initiative teaching, representation and reasoning. The system will understand what a student says and emulate human responses. An existing natural language speech (input and output) dialogue system will be integrated with an existing intelligent tutoring system. The existing mixed-initiative language and planning system, TRAINS-96, enables humans and computers to work together in a tightly coupled way to solve problems that neither alone could manage, within the context of command and control (Ferguson and Allen 1998; Allen et al. 1995). It helps a manager solve routing problems in a simple transportation domain. Tutorial discourse is different from other types of discourse, including problem solving and advisory discourse, in that it should facilitate student understanding rather than help formulate a plan of action. In a cooperative problem-solving situation both machine and human try to solve the same problem using their respective knowledge and abilities. The system we will build should understand domain knowledge and reason about the state of the student's presumed knowledge. Questions in the tutorial dialogue will be used to diagnose student understanding and then provoke knowledge construction. Another prominent characteristic is that the student may not be familiar with the concepts being discussed or possibly even the vocabulary being used. Existing intelligent tutors can analyze student behavior and customize their responses (e.g., generate a specific learning opportunity or produce an appropriate hint) based on inferences about student knowledge and prior actions (Eliot and Woolf 1995, 1996; Beck et al. 1997; Beal et al. 1998). In several cases these intelligent tutors are used in higher education or grade schools with thousands of students. Comparison to Existing Dialogue Tutors TheTRAIN-Tutor system will differ from other dialogue systems in that it will accept and generate spoken language input and output. Therefore it will be particularly useful for tutoring dialogue and helpful for younger students. Additionally, it will be knowledge-based, having full natural language parsing, understanding, planning and generation systems. It differs from Auto-Tutor, based on Lexical Semantic Analysis (LSA), primarily in that it keeps track of the history of the student and does not require large numbers of training. For each new domain, LSA requires numerous essays or dialogues to be encoded and compared with input students. For example to encode 3 topics, 175 pages of student protocols or 1,000 items per topic might be encoded. LSA does not handle dialogue, does not track focus of attention and does not resolve anaphoras. Circsim-Tutor, another dialogue tutor, only handled small input sentences and relied on shallow processing (e.g., classifying the input sentences by recognizing key words). It was effective only for evaluating content based on inclusion of relevant vocabulary. Atlas-Andes has full pedagogical knowledge and possibly deeper natural language knowledge than Circsim, but it does not yet handle complex dialogues. Our goal is to support a deeper level of analysis and to identify complex relationships between concepts in longer student answers. We intend to support collaborative dialogue between the student and tutor, to understand sub dialogues and to recognize when new beliefs are adopted by the student. We plan to use discourse coherence principles, that is, to structure and take advantage of the focus of attention as it shifts through the conversation. Tracking the focus of attention is necessary for generating coherent student tutor dialogue and for pronoun or anaphora interpretation and generation. Dialogue Tutoring for Arithmetic Problems The TRAINS Dialogue system will be integrated with AnimalWatch, which provides adaptive and effective mathematics instruction for grade school children (Arroya et al. 1997; Beck et al. 2000). The dialogue system will speak to the student when he or she has trouble solving a problem. Currently the tutor initiates an interaction and provides customized hints to help the student work through a problem. The student model maintains an accurate assessment of the studentÕs strengths and weaknesses, has a record of his or her cognitive development and helps 1) generate appropriately difficult problems and 2) respond to the studentÕs errors with feedback tailored to her or his needs. Machine learning techniques are used to select the problem and help, based on the experience of hundreds of previous users, (Beck et al. 2000). Multimedia is used judiciously to engage the student by animating key concepts and providing interactive manipulables based on those used by classroom teachers. Math problems are not ÒcannedÓ or pre-stored. Rather, hundreds of problem templates are used to generate novel problems Òon the fly.Ó Evaluation studies showed that students responded differently to help and feedback. For example, highly adaptive feedback is especially important to girls (Arroya et al. 1999). Also, both boys and girls of lower cognitive development need more hints to solve problems (Arroya et al. 1999). However, there was a strong relation between a girlÕs cognitive development and her view of how helpful different hints were: hints that were highly interactive (i.e., structured) were rated as significantly more helpful than less interactive hints, and were more effective (i.e., were followed by fewer errors in subsequent problems). No such relation was found for boys. Overall, results indicate that not only is adaptive feedback especially important for girls, certain specific types of feedback are preferred by girls, whereas boys do not appear to show such consistent preferences. Several evaluation studies showed significant improvements in attitudes towards math (confidence, value, liking) after students worked with AnimalWatch (Beck et al. 1999). The TRAINS-Tutor will produce three dialogue types: Diagnosis Dialogue. If the student provides a wrong answer, the Tutor may choose to change the direction of the dialogue, to identify missing knowledge and provide an explanation. This dialogue will be tailored to the problem on which the student is working. Tutor: Do you know how to add fractions? Student: I think so. You add the tops and then you add the bottoms. Tutor: Let's look at this more closely. The first step is to check that you have equivalent denominators. Do you have equivalent denominators? Student: Why do I need equivalent denominators? Tutor: The denominators must be equal to ensure that you are adding similar quantities. Hint Dialogue. The tutor will select from a large supply of hints, those most appropriate for a particular diagnosed problem. The hint will be chosen based on student gender, cognitive development, response to earlier hints, problem history, etc. Tutor: Did you find the Least Common Multiple? Student: Do you mean when the denominators are equal? Tutor: Let me explain Least Common Multiple with an animation. Summary Dialogue. After the tutor has given the student several hints, it will provide a summary. This is to ensure that student has correct information even if he or she provided the correct answer by simply following hints without understanding the procedures. Student answers might be correct, wrong, near-miss or unrelated. The NLP data base can already handle affirmative, negative answers as well as partial misunderstanding, thanks to its representation of natural language. All idiomatic expressions, such as "I haven't got a clue" must be added to the lexicon. TRAINS can handle anaphora and can generally interpret the user's input within context. Thus it can recognize what part of the problem a student is referring to but from the expert point of view.